Content optimization system and method

ABSTRACT

Disclosed herein is a statistical approach, a win share approach, used to assign a win share value to content items. User interaction with content items is tracked, and a win share value is assigned to content items in response to a “winning” action performed by a user. Win shares associated with content items are used to identify content items that are to be presented, and can further be used to identify an optimal presentation, e.g., layout, presentation frequency, etc., of content items that is to be presented.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional PatentApplication No. 60/984,277, filed Oct. 31, 2007 and entitled “CONTENTOPTIMIZATION SYSTEM AND METHOD”, the contents of which are incorporatedherein by reference.

TECHNICAL FIELD

This disclosure relates to optimization of content in contentpresentation, and more particularly to use of a win share associatedwith content items to optimize content presentation.

BACKGROUND

A constant challenge faced by providers of content, e.g., a portal,website provider, etc., is identifying content to be displayed to auser. A user that visits a web site, or a web page displayed at the website, typically scans the page for something of interest. If the userdoes not find something of interest, the user will typically leave theweb page, or the web site altogether.

Conventional approaches manage keywords and a link structure within thewebsite to ensure good traffic from search engines.

SUMMARY

The present disclosure seeks to address failings in the art and toprovide a system and method for optimizing content, and the presentationof content.

Embodiments of the present disclosure can be used in determining whatcontent to display, and where content should be placed on the web page,for example. By way of some non-limiting examples, content can includenews, multimedia, research, advertisements, search results, web pages,etc. content. By way of a further non-limiting example, content canoriginate from the users themselves, or can be provided by anothercontent provider. Embodiments can be used to identify content to bepresented, e.g., engaging user-generated content. In addition toidentifying content to be presented, embodiments of the presentdisclosure can be used to identify an optimal presentation for content,e.g., so that content is presented in way that it engages users.

Embodiments use a statistical approach to assign a win share value tocontent items. In accordance with one or more such embodiments, userinteraction with content items is tracked, and a win share value isassigned to content items in response to a “winning” action performed bya user. Win shares associated with content items can be used to identifycontent items that are to be presented, and can further be used toidentify an optimal presentation, e.g., selection, layout, frequency,etc., of content items.

In accordance with one or more embodiments, a win share approach is usedto optimize content, e.g., display content available via a userinterface, such as that provided via a web site. Embodiments of thepresent disclosure collect information associated with user interactionswith the user interface, which can include user selections and otheruser interaction with the user interface. In accordance with one or moresuch embodiments, a portion of the user interaction, e.g., interactionwith a subset of the users that interact with the user interface, isused to optimize content, so as to identify, e.g., filter, content,e.g., engaging content, from other content, and then present, e.g., pushout, the identified content to the entire population via one or more webpages offered via the user interface.

In accordance with an aspect of the present disclosure comprise asystem, or method, that determines a win share for a plurality ofcontent items, prioritizes the content items based on the win sharedetermined for each of the plurality of content items, and presents oneor more content items from the plurality of content items based on thedetermined win shares. In accordance with one or more of suchembodiments, the presentation of content items includes determining afrequency with which the one or more content items are presented basedon the determined win shares.

In accordance with another aspect of the present disclosure, a system,or method, determines a win share by tracking user interaction with oneor more of a plurality of content items, recognizing a predeterminedinteraction with a user, identifying content items from the plurality ofcontent items with which the user interacted preceding the predeterminedinteraction, and assigning a win share to each of the identified contentitems. In accordance with one or more aspects of the present disclosure,the function distributes the win share evenly across the identifiedcontent items. In accordance with other aspects of the presentdisclosure, the function distributes the win share unevenly, based on agiven weighting, across the identified content items. The weighting canbe based on factors such as a timing of the user's interaction with acontent item, one or more characteristics associated with the contentitems and/or the user.

DRAWINGS

The above-mentioned features and objects of the present disclosure willbecome more apparent with reference to the following description takenin conjunction with the accompanying drawings wherein like referencenumerals denote like elements and in which:

FIG. 1 provides examples of an activity-based web log and time-based weblog for use in accordance with one or more embodiments of the presentdisclosure.

FIG. 2 provides an example of an ordering of content IDs associated withcontent items, the ordering being based on the win share valuesassociated with the content IDs.

FIG. 3 provides an example of content identification using win sharesdetermined using a subset of an entire user population in accordancewith one or more embodiments of the present disclosure.

FIG. 4 provides an example of a data structure, in the form of adatabase table, which can be used to generate win shares in accordancewith one or more embodiments of the present disclosure.

FIG. 5, which comprises FIGS. 5A to 5C, provides examples of databasequeries for use in accordance with one or more embodiments of thepresent disclosure.

FIG. 6, which comprises FIGS. 6A and 6B, provides results of humanjudgments corresponding to content judged in accordance with embodimentsof the present disclosure.

FIG. 7 illustrates some components that can be used in connection withone or more embodiments of the present disclosure.

FIG. 8 provides an exemplary process flow to optimize content inaccordance with one or more embodiments of the present disclosure.

DETAILED DESCRIPTION

In general, the present disclosure includes a content optimizationsystem, method and architecture.

Certain embodiments of the present disclosure will now be discussed withreference to the aforementioned figures, wherein like reference numeralsrefer to like components.

Embodiments of the present disclosure use a win share approach tooptimize content. A win share method can be explained with reference toa baseball team, and selection of players for a team. Assume that nplayers are to be picked for a team from a collection of m players, andthat there are to be N games played before the final team is selected.Various combinations of n players can be selected for each of the Ngames. The players that are a part of a team that wins a game can beassigned a share of a win, and a player's win share can be used toselect players for another team. To illustrate, every time a team wins,each one of the n players on the team is assigned 1/n^(th) of a share ofthe team's win. At the end of N games, an aggregate win share isdetermined for each player based on that player's win share for the Ngames. The players can be sorted in descending order based on theiraggregate win share value. To pick a new team from the m players, thetop n players can be selected from the ordered list. This method rewardsplayers who were associated with a team win by giving them an equalshare of the win, simultaneously it penalizes users who were notinvolved in a win, since a player that plays for a losing team wouldhave a win share that is less than another player that plays on awinning team. The book entitled Win Shares by Bill James is incorporatedherein by reference.

A win share mechanism can be used to optimize content displayed using auser interface, e.g., web content, where the web content, or contentitems, are the players, and a win comprises a winning user action. e.g.,a desired user action with a website or the web content. In accordancewith one or more embodiments of the present disclosure, user websiteactivity is collected and stored in web logs. The web logs are convertedto a time log of activity on the website. FIG. 1 provides examples of anactivity-based web log 102 and a time-based web log 104 for use inaccordance with one or more embodiments of the present disclosure. Inaccordance with such embodiments, the activity-based web log 102 is usedto create the time-based log 104. It should be apparent that theactivity-based and time-based web logs 102 and 104, respectively, can becreated concurrently, or that the time-based web log 104 can be createdalone, without the activity-based web log 102, e.g., in a case that anactivity-based log 102 is not already being generated. It should beapparent that the examples shown in FIG. 1 can comprise an excerpt froman activity-based log 102 and/or a time-based log 104, with additionalinformation being stored in either log.

In the example shown in FIG. 1, the activity-based log 102 includes rowentries 106A to 106J, each of which includes, without limitation, timefield 108, cookie field 110 and content identifier (ID) field 112. Toillustrate, each row 106 in the activity-based log 102 corresponds to auser's interaction with the website/web page, e.g., content presented bythe website or web page. In such a case, the time field 108 identifies atime for the user activity, the cookie field 110 identifies the user,and the content field 112 contains a content item with which the userinteracted, as identified by a content ID, Q_(N), which can be used toidentify the particular item of content, group of content items, etc. Alog can span a period of time, such as one or more years, months, days,hours, sessions, etc.

While the activity-based log 102 shows particular examples of values forthe time, user identification and content identification, it should beapparent that any manner of identifying time, user and content can beused with embodiments of the present disclosure. In addition and whiletime is indicated across all of the users, it should be apparent thattime can be indicated relative to each user, i.e., each user has its owntime frame. Similarly, in accordance with one or more embodiments, usersare identified based on a unique user identifier (ID) contained in acookie, or other store, maintained at the user's computer or otherstorage location, e.g., a server. Finally, any technique can be used toidentify content, such as a uniform resource locator, or other contentidentifier.

In accordance with one or more embodiments, data contained in theactivity-based log 102 is used to create a time-based log 104, whichidentifies a chain of activity for a given user, or a “chain of content”that a user views, or otherwise interacts with, on the website.Referring again to FIG. 1 and for the sake of a non-limiting example, itis assumed that users A, B, C, D “interacted with” content as shown inthe activity-based log 102, such as content selection, mouse over, etc.,for example.

The activity-based log 102 can be used to generate the time-based log104 also shown in FIG. 1. In accordance with one or more embodiments,the time-based log 104 has entries 114, each of which corresponds to agiven user. With reference to the time-based log 104 example shown inFIG. 1, entry 114A corresponds with user A and indicates that user Ainteracts with content item Q1 and Q2, entry 114B corresponds to user Bthat interacts with content items Q2, Q1, Q3 and Q4, entry 114Ccorresponds with user C that interacts with content items Q1, Q2 and Q3,and entry 114D corresponds with user D that interacts with content itemsQ1. The time-based log 104 provides a time line 116 of a user'sactivity, which identifies an order in which the user interacted withthe content items. For example and as shown in entry 114B of thetime-based log 104, user B interacts with content Q2, then interactswith content Q1, then Q3 and then Q4 at times along time line 116.

In accordance with one or more embodiments of the present disclosure,the content items correspond to the “players”, for which a win share isto be identified. In accordance with such embodiments, a win can beanything determined to be significant. A winning action can be, forexample, anything that the stakeholders of the website are interested in“driving up” using the content, such as registering users with thewebsite. By way of a non-limiting example, a win can be a user viewing acontent item, or interacting with something that corresponds with thecontent item. By way of a further non-limiting example, the interactioncan be mouse over, click through or other interaction with the contentor an item associated with the content. By way of yet anothernon-limiting example, a winning action can comprise a user providinginformation as part of a user profile creation, e.g., user registration,operation. Referring to the illustrative activity-based log 102 shown inFIG. 1, a winning action occurs in correspondence with the times 7, 9and 10, corresponding to users C, B and D, respectively. It should beapparent that while the winning action could be indicated in a separateentry in the activity-based log 102, for the sake of simplicity, it isidentified in connection with the last content accessed by the userprior to the occurrence of the winning action. In a case in which thewinning action has its own entry in the activity-based log 102, thecontent field could be used to identify the winning activity, e.g.,creating a profile.

In accordance with embodiments of the present disclosure, the winning“team” comprises the content items, e.g., the chain of content items,that the user interacted with, e.g., viewed prior to a point in timethat the user initiated the “winning action”, e.g., the user created theprofile. In accordance with or one more such embodiments, it is assumedthat content items with which the user interacted prior to initiatingthe winning action contributed to the winning action. By way of anon-limiting example and in accordance with one or more suchembodiments, if a value of 1 is associated with a winning action, all ofthe content items that contributed to the winning action share in afraction, 1/m, where m represents the number of content items identifiedas contributing to the win. In this example, the win is divided equallyamong all of the content items that contributed to the win. It should beapparent that other distributions of the win can be used. For example,certain content items, e.g., newer content items, older content items,content items more recently viewed, etc., can be given a greater, orlesser, portion of the win.

FIG. 2 provides win share examples for the example logs shown in FIG. 1in accordance with one or more embodiments of the disclosure. Matrix 200includes rows 202A to 202D, which correspond to content items Q2, Q1, Q3and Q4, and columns 204A to 204D, which correspond to users A, B, C andD. Column 204E shows an aggregate win share value determined using theexample provided in FIG. 1.

As can be seen from the activity-based log 102 and time-based log 104 ofFIG. 1, while user A interacted with Q1 and Q2 content items, user A didnot perform a winning action. In accordance with one or moreembodiments, since user A did not perform a winning action, user A doesnot contribute to the win share value of any of the content items,including the content items with which user A interacted. This result isshown in column 204A of FIG. 2, which includes a zero win share for inall of the rows 202A to 202D. User B, on the other hand, interacted withcontent items Q1, Q2, Q3 and Q4 prior to a winning action. In accordancewith at least one embodiment, content items Q1, Q2, Q3 and Q4 share¼^(th) of the win, in a case that the content items share equally in thewin. As shown in column 204B and rows 202A to 202D, each content item isgiven a ¼ share of the win.

Similarly and since user C interacted with three content items, i.e.,Q1, Q2, Q3, prior to a winning action, these three content items sharein the win. As shown in column 204C and rows 202A to 202C, each item isgiven ⅓^(rd) of the credit for the win, in a case that the content itemsshare equally in the win. In the case of user D, since user D viewed onecontent item, Q1, prior to the winning action, the one content item getsthe entire share of the win. As shown in column 204D and row 202A,content item Q1 is assigned a win share of 1. The win shares distributedto each of the content items based on users B, C and D interaction withthe content and winning action are aggregated to determine an aggregatedwin share for a given content item. In the example, as shown in column204E, the content items Q1, Q2, Q3 and Q4 have aggregate win shares 204Eshown in rows 202A to 202D of 19/12, 7/12, 7/12 and ¼, respectively.

In accordance with one or more embodiments, the content items can beordered using their aggregate win share values. For example, the contentitems can be sorted in descending order of their aggregate win sharevalue. FIG. 2 provides an example of an ordering of content items, andtheir associated content IDs, the ordering being based on the win sharevalues associated with the content items. In the example shown in FIG.2, content item Q1 is ranked above content items Q2 and Q3, which sharesecond place, followed by content item Q4.

In accordance with one or more embodiments, the entire user communitycan be used to determine win shares. Alternatively, a subset of the usercommunity can be used. In accordance with one or more embodiments, afeedback loop can be used, wherein new content is tested against theentire user community or a small percentage, e.g., 5%, of the populationdetermined to fairly represent the entire population. In a case that arepresentative user sample is used, the win shares can be computed usingthe representative user sampling. The resulting win shares can be usedto identify the content items from the new content items that are to beprovided to the entire population.

FIG. 3 provides an example of content identification using win sharesdetermined using a subset of an entire user population in accordancewith one or more embodiments of the present disclosure. In the exampleof FIG. 3, win shares determined using a representative user sample 302can be used to determine a frequency with which content is presented tothe user population. The representative user sample 302 can comprise apercentage of the users determined to represent the entire population ofusers 306. In the example shown in FIG. 3, new content items 304, e.g.,Q1, Q2 and Q3, are shown to the representative user sample 302. Winshares are determined for the new content items 304 exposed to the usersampling 302 in accordance with one or more embodiments of the presentdisclosure. In the example of FIG. 3, the computed win shares can beused to determine an ordering 308 of the content items 304. In theexample, content item Q3 has a greater win share than the win sharedetermined for content item Q1, which has a greater win share thancontent item Q2.

In accordance with one or more embodiments, a frequency with which thenew content items are presented to the entire user population 306 can bedetermined, such that content item Q3 is presented to the userpopulation 306 more frequently than is content item Q1, which ispresented to the user population 306 more frequently than content itemQ2. By way of a non-limiting example and in accordance with one or moreembodiments, content with high win shares can be shown to a userpopulation 306 based on win shares determined by a representative usersample 302, thus greatly improving chances of invoking winning actionsfrom the user population 306. In accordance with such embodiments, winshares can be determined for new content 304 without the need to involveall of the user population 306, e.g., in a case that the user sample 302is 5% of the user population 306, 95% of the user population 306 neednot be involved in judging the new content 304 and/or determining thefrequency with which new content 304 is presented. Using the user sample302, win shares can be determined for new content 304, and new content304 with higher win shares can be shown more frequently than new content304 with lesser win shares.

Use of a sampling of users in accordance with one or more embodiments,is advantageous, among other reasons, as it provides an ability toreduce the resources used to determine win shares. For example, datastructures and storage amount needed to store log information, win shareinformation, etc. can be reduced in a case that a user sample is used todetermine win shares. In addition, the data structures can berepresented as simple data structures, e.g., as relational databasetables, which can be queried using a structured query language, SQL,thereby avoiding a need to have complex applications running to filtercontent.

FIG. 4 provides an example of a data structure, in the form of adatabase table, which can be used to generate win shares in accordancewith one or more embodiments of the present disclosure. In the exampleof FIG. 4, a table 400 is created using a sampling, e.g., a sample sizecomprising 5% of the user population, of users who are exposed to newcontent over a given period of time, a given day. The table stores anevent log of how the users in the user sample interacted with the website up until they left the web site or performed a winning action,e.g., created a profile. The rows 402 of the table 400 are created eachtime a user interacts with a content item. Each row 402 has a time stampfield 404A, user identification information field 404B, content itemidentification field 404C and a profile created field 404D. The fields404 in each row 402 identify a time, a user, a content item, and whetheror not the user created, or activated, a profile.

The table 400 can be queried to identify a list of content IDscorresponding to content items with which a user interacted, e.g.,viewed, prior to performing a winning action, e.g., activating aprofile. FIG. 5, which comprises FIGS. 5A to 5C, provides examples ofdatabase queries for use in accordance with one or more embodiments ofthe present disclosure. A list of the “top ranked” content IDs, e.g.,the content IDs having the largest win shares, can be determined usingthe queries shown in FIG. 5 and the database table shown in FIG. 4.

The query shown in FIG. 5A retrieves a list of content IDs a given userinteracted with on a website prior to the user initiating a profileactivation operation. The query shown in FIG. 5B retrieves a count ofcontent items with which the user interacted on the website prior to theuser initiating a profile activation operation. The query shown in FIG.5C is used to compute the win shares for the content items identified inthe query shown in FIG. 5A using the content determined using the queryshown in FIG. 5B.

In one exemplary embodiment of the present disclosure, data is collectedfor an application with which user interacts via the web. Theapplication allows a user population to post questions and answers toposted questions. In accordance with this embodiment, the content cancomprise the question or the answers to the questions, orquestion-answer pairs, and the winning action corresponds to a usercreating a profile. Once a user creates a profile, the user is able tosubmit questions and/or answers to the questions, for example. Usersession data is collected to determine the win shares of various contentIDs in accordance with one or more embodiments disclosed herein.

A small sample of the content for which win shares were determined wasalso made the subject of a blind test, which involved human editors, orjudges, providing their judgment concerning the content in the sample.The editors had no knowledge of the win shares associated with thecontent for which they were asked to provide their judgments. Theeditors were asked to rate each item of content using a three pointscale of 1, 2 and 3, where 1 represents content judged to be excellent,2 represents content judged to be mediocre content, and 3 representscontent judged to be least desirable, e.g., least informative,questionable content, and/or of little importance or interest.

FIG. 6A provides an example of results of human judgments correspondingto content judged in accordance with embodiments of the presentdisclosure. Each row 602 in table 600 includes a judges' scoring field604A, number of content items scored field 604B, an average win sharefield 604C and standard deviation field 604D. As can be seen from thetable 600 shown in FIG. 6A, the average win shares 604C for contentitems correlate with the judges' scores 604A. With reference to row602A, the content items given a rating of 1 by the judges have anaverage win share 604C that is greater than the average win shares 604Cof those content items given a rating of either 2 or 3 by the judges inrows 602B and 602C, respectively. The content items judged to have arating of 1 by the judges had an average win share of 6.97, with astandard deviation of 11.52. This is in contrast to the content itemsrated 2 and 4, i.e., the 49 and 96 content items judged by the judges,which have average win shares of 4.75 and 1.57, respectively.

As can be seen from the standard deviations, win share scores varied inthis example. This can be attributed to the type of content tested,i.e., questions and answers that included user-generated content. Suchcontent can be of low quality, with several grammatical and spellingmistakes, which can invoke a poor response and content generationactivity from the user community.

FIG. 6B provides a graphic that shows a cumulative distribution of winshares determined in accordance with one or more embodiments. In thegraph shown in FIG. 6B, the x-axis corresponds to win share scores forthose question-answer pairs rated 1, 2 or 3 by the human judges on alogarithmic scale. The y-axis corresponds to a cumulative fraction, orpercentage of content items having both a win share score and judges'scoring. In the table shown in FIG. 6B, the x-axis corresponds to winshare scores associated with question-answer pairs rated as 1, 2 or 3 bythe judges on a logarithmic scale.

Some items of interest include:

-   -   The win share scores at the 50% mark of on the y-axis for        question-answer pairs rated 3 by the human judges, point 614C,        is significantly below the 50% marks, points 614A and 614B        (respectively), corresponding to question-answer pairs rated 1        and 2 by the judges. That is, fifty percent of the content items        rated 3 by the judges have a win share score below 0.01, while        fifty percent of the content items rated 1 or 2 by the judges        have win share scores above 1.    -   Approximately 45% of the question-answer pair content in this        example, that was rated 2 by the human judges has win share        scores less than 0.01.    -   The maximum win share score of question-answer pairs rated 3 is        27.8 and more than 5% of those rated 1 are above that win share        score.

In accordance with embodiments of the present disclosure, win sharecontent optimization provides an effective mechanism to identify optimalcontent, e.g., so as to identify content that is likely to increase auser's engagement with a website.

Content optimization using win shares as described herein can be used incombination with other filtering mechanisms. For example, it is possiblethat win shares associated with content items can be used to identifycontent items, e.g., question-answer pairs, with high win shares. Insome cases, high win share scores may correspond with abusive content.For example, abusive content might tend to enrage users and make themwant to engage, which can result in high win shares for such content.Embodiments of the present disclosure can be used in combination withadditional filtering mechanisms to avoid such content beingre-presented, e.g., use human judgment in a review of content having winshares considered to be abnormally high and/or low.

FIG. 7 illustrates some components that can be used in connection withone or more embodiments of the present disclosure. In accordance withone or more embodiments of the present disclosure, one or more computingdevices, e.g., one or more servers 702, are configured to optimizecontent using win shares determined for content items. The one or moreservers 702 can serve content to user computers 704 using a browserapplication via a network 706, and track user interaction with thecontent and winning actions in the browser application. Such userinteractions can be stored in a data store 708, e.g., as logs stored inthe data store 708. The logged interactions can be used to determine winshare amounts for the content. The determined win shares can be used toprioritize content based on win shares associated with the content, suchthat content that has high win shares can be selected to be served tothe user computers 704. Alternatively, a frequency with which a contentitem is served to the user computers 704 can be determined based on acontent items win share.

The user computer 704 can be any computing device, including withoutlimitation a personal computers, personal digital assistant (PDA),wireless device, cell phone, internet appliance, media player, hometheater system, and media center, or the like. For the purposes of thisdisclosure a computing device includes a processor and memory forstoring and executing program code, data and software, and may beprovided with an operating system that allows the execution of softwareapplications in order to manipulate data. A computing device such asserver 702 and the user computer 704 can include one or more processors,memory, a removable media reader, network interface, display andinterface, and one or more input devices, e.g., keyboard, keypad, mouse,etc. and input device interface, for example. One skilled in the artwill recognize that server 702 and user computer 704 may be configuredin many different ways and implemented using many different combinationsof hardware, software, or firmware.

In accordance with one or more embodiments, the server 702 makes a userinterface available to a user computer 704 via the network 706. The userinterface made available to the user computer 704 can include contentitems selected for the user interface using win shares associated withthe content items. In accordance with one or more embodiments, server702 makes a user interface available to a user computer 704 bycommunicating a definition of the user interface to the user computer704 via the network 706. The user interface definition can be specifiedusing any of a number of languages, including without limitation amarkup language such as Hypertext Markup Language, scripts, applets andthe like. The user interface definition can be processed by anapplication executing on the user computer 704, such as a browserapplication, to output the user interface on a display coupled, e.g., adisplay directly or indirectly connected, to the user computer 704.

In an embodiment the network 706 may be the Internet, an intranet (aprivate version of the Internet), or any other type of network. Anintranet is a computer network allowing data transfer between computingdevices on the network. Such a network may comprise personal computers,mainframes, servers, network-enabled hard drives, and any othercomputing device capable of connecting to other computing devices via anintranet. An intranet uses the same Internet protocol suit as theInternet. Two of the most important elements in the suit are thetransmission control protocol (TCP) and the Internet protocol (IP).

It should be apparent that embodiments of the present disclosure can beimplemented in a client-server environment such as that shown in FIG. 7.Alternatively, embodiments of the present disclosure can be implementedon a standalone computing device, such as the server or computer shownin FIG. 7.

FIG. 8 provides an exemplary process flow to optimize content inaccordance with one or more embodiments of the present disclosure. Atstep 802, content items, or type of content items (e.g., multimedia,question, answer, question-answer pairs, search results, etc.), andwinning action(s) are identified. At step 804, user interactions withthe content items and winning actions are tracked and stored in a log.At step 806, the user logs are examined to identify those users thatperformed a winning action and the winning actions performed by suchusers. At step 808, a determination is made whether or not all of theusers and winning actions identified have been processed. If so,processing ends at step 818.

If it is determined that one or more users or winning actions remain tobe processed, processing continues at step 810 to get the next user tobe processed. At step 812, the log(s) are examined to identify thecontent that led up to the user's winning action. A win share isdetermined for each of the identified content items at step 814. At step816 the win share for a content item is added to the win share balancefor the content item. By way of a non-limiting example, a win share forthe i^(th) content item, Ψ(Q_(i)), can be determined using the followingequation:Ψ(Q _(i))←Ψ(Q _(i))+1/N,  Eq. 1

where N is the identified number of content items that led up to thewinning action, and 1/N is the content item's share of the currentwinning action being processed. Processing continues at step 808 toprocess any remaining users and winning actions.

In the above equation 1, a content item's win share is a function of theset of content items associated with a winning action. The equation canbe written more generally as:Ψ(Q _(i))←F(Ψ(Q _(i)),Ω),  Eq. 2

where Ω is the set of content items, Q_(i), associated with a winningaction. One example of the function F(Ψ(Q_(i)),Ω) is Ψ(Q_(i))+1/N. Otherexamples of such a function include but are not limited to using aweighted fraction 1/N, where the weighting is based on such factors asprimacy and/or recency. For example, newer content items can be given agreater, or lesser, share of a win. By way of a further non-limitingexample, content items that the user interacted with closer in time tothe winning action than other content items can be given a greater, orlesser, share of the win. A win share can be a function of the usersthemselves, and knowledge about the users that interact with thecontent. For example, a win share can be increased, or decreased, basedon knowledge about the users, such as consistency and/or reliability ofinput, or known characteristics associated with users. As yet a furthernon-limiting example, a win share can be adjusted based on the contentitems themselves, and or an additional filtering mechanism (e.g., humanjudgment information).

For the purposes of this disclosure a computer readable medium storescomputer data, which data can include computer program code executableby a computer, in machine readable form. By way of example, and notlimitation, a computer readable medium may comprise computer storagemedia and communication media. Computer storage media includes volatileand non-volatile, removable and non-removable media implemented in anymethod or technology for storage of information such ascomputer-readable instructions, data structures, program modules orother data. Computer storage media includes, but is not limited to, RAM,ROM, EPROM, EEPROM, flash memory or other solid state memory technology,CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetictape, magnetic disk storage or other magnetic storage devices, or anyother medium which can be used to store the desired information andwhich can be accessed by the computer.

In accordance with one or more embodiments, a user interface with whichcontent items is made available to a user, e.g., the user's computingdevice, via a network such as the Internet or other network, and contentitems are made available via the user interface based on win sharesassociated with one or more of the content items. By way of anon-limiting example, a content item's win share can be used todetermine whether or not to make the content item available, thefrequency with which the content item is made available, and/or adisplay order for the content items in the user interface.

Those skilled in the art will recognize that the methods and systems ofthe present disclosure may be implemented in many manners and as suchare not to be limited by the foregoing exemplary embodiments andexamples. In other words, functional elements being performed by singleor multiple components, in various combinations of hardware and softwareor firmware, and individual functions, may be distributed among softwareapplications at either the client or server or both. In this regard, anynumber of the features of the different embodiments described herein maybe combined into single or multiple embodiments, and alternateembodiments having fewer than, or more than, all of the featuresdescribed herein are possible. Functionality may also be, in whole or inpart, distributed among multiple components, in manners now known or tobecome known. Thus, myriad software/hardware/firmware combinations arepossible in achieving the functions, features, interfaces andpreferences described herein. Moreover, the scope of the presentdisclosure covers conventionally known manners for carrying out thedescribed features and functions and interfaces, as well as thosevariations and modifications that may be made to the hardware orsoftware or firmware components described herein as would be understoodby those skilled in the art now and hereafter.

While the system and method have been described in terms of what arepresently considered to be the most practical and preferred embodiments,it is to be understood that the disclosure need not be limited to thedisclosed embodiments. It is intended to cover various modifications andsimilar arrangements included within the spirit and scope of the claims,the scope of which should be accorded the broadest interpretation so asto encompass all such modifications and similar structures. The presentdisclosure includes any and all embodiments of the following claims.

What is claimed is:
 1. A method comprising: tracking, via a computingdevice, user interaction with a user interface; recognizing, via thecomputing device, a predetermined interaction of a user with the userinterface, said predetermined interaction identifies a time of thepredetermined interaction, the user and a content item with which theuser interacted; identifying, via the computing device, a set of contentitems from a plurality of content items with which the user interactedprior to the predetermined interaction; and determining, via thecomputing device, a share of a winning score to each identified contentitem in said set of content items based on a number of content items insaid set, said determination further comprising assigning said share toeach identified content item.
 2. The method of claim 1, whereinassigning a share of a winning score to each identified content itemfurther comprising: distributing the winning score evenly across two ormore identified content items.
 3. The method of claim 1, whereinassigning a share of the winning score to each identified content itemfurther comprising: distributing the winning score unevenly across twoor more identified content items.
 4. The method of claim 3, whereindistributing the winning score unevenly across two or more identifiedcontent items comprising: determining a weight for each of theidentified content items that is based on a timing of the user'interaction with the identified content item; and applying theidentified content item's weight to the winning score to determine a winshare for the identified content item.
 5. The method of claim 4, whereindistributing the win share unevenly across two or more identifiedcontent items comprising: determining a weight for each of theidentified content items that is based on a newness characteristicassociated with the identified content item; and applying the identifiedcontent item's weight to a win share value to determine a win share forthe identified content item.
 6. The method of claim 1, furthercomprising: identifying at least one of the plurality of content itemsto make available via the user interface based on a win share associatedwith the at least one of the plurality of content items.
 7. The methodof claim 1, further comprising: determining a placement in the userinterface of at least one of the plurality of content items based on awin share associated with the at least one of the plurality of contentitems.
 8. The method of claim 1, further comprising: determining anordering of two or more of the plurality of content items in the userinterface based on a win share associated with each of the two or morecontent items.
 9. The method of claim 1, further comprising: determininga frequency with which at least one of the plurality of content items ismade available via the user interface based on a win share associatedwith the at least one of the plurality of content items.
 10. The methodof claim 1, wherein the user is a member of a sample of a userpopulation, the method further comprising: making one or more of theplurality of content items available to the user population based on winshares assigned using the user sample.
 11. A method comprising: making,via a computing device, a user interface available over a network;making, via the computing device, a plurality of content items availablevia the user interface, at least one content item of the plurality beingmade available based on a win share assigned to the content item,wherein a win share is determined by: tracking, via the computingdevice, user interaction with the user interface; recognizing apredetermined interaction of a user with the user interface, saidpredetermined interaction identifies a time of the predeterminedinteraction, the user and a content item with which the user interacted;identifying, via the computing device, a set of content items from theplurality of content items with which the user interacted prior to thepredetermined interaction; and determining, via the computing device, ashare of a winning score to each identified content item in said set ofcontent items based on a number of content items in said set, saiddetermination further comprising assigning said share to each identifiedcontent item.
 12. The method of claim 11, further comprising:determining a placement in the user interface of at least one of theplurality of content items based on a win share associated with the atleast one of the plurality of content items.
 13. The method of claim 11,further comprising: determining an ordering of two or more of theplurality of content items in the user interface based on a win shareassociated with each of the two or more content items.
 14. The method ofclaim 11, further comprising: determining a frequency with which atleast one of the plurality of content items is made available via theuser interface based on a win share associated with the at least one ofthe plurality of content items.
 15. A non-transitory computer-readablestorage medium tangibly encoded with computer-executable program code,that when executed by a computing device, performs a method comprising:track user interaction with a user interface; recognize a predeterminedinteraction of a user with the user interface, said predeterminedinteraction identifies a time of the predetermined interaction, the userand a content item with which the user interacted; identify a set ofcontent items from a plurality of content items with which the userinteracted prior to the predetermined interaction; and determining ashare of a winning score to each identified content item in said set ofcontent items based on a number of content items in said set, saiddetermination further comprising assigning said share to each identifiedcontent item.
 16. The non-transitory computer-readable storage medium ofclaim 15, the program code to assign a share of a winning score to eachidentified content item further comprising code to: distribute thewinning score evenly across two or more identified content items. 17.The non-transitory computer-readable storage medium of claim 15, theprogram code to assign a share of the winning score to each identifiedcontent item further comprising code to: distribute the winning scoreunevenly across two or more identified content items.
 18. Thenon-transitory computer-readable storage medium of claim 17, the programcode to distribute the winning score unevenly across two or moreidentified content items further comprising code to: determine a weightfor each of the identified content items that is based on a timing ofthe user's interaction with the identified content item; and apply theidentified content item's weight to the winning score to determine a winshare for the identified content item.
 19. The non-transitorycomputer-readable storage medium of claim 18, the program code todistribute the win share unevenly across two or more identified contentitems further comprising code to: determine a weight for each of theidentified content items that is based on a newness characteristicassociated with the identified content item; and apply the identifiedcontent item's weight to a win share value to determine a win share forthe identified content item.
 20. The non-transitory computer-readablestorage medium of claim 15, the program code further comprising code to:identify at least one of the plurality of content items to makeavailable via the user interface based on a win share associated withthe at least one of the plurality of content items.
 21. Thenon-transitory computer-readable storage medium of claim 15, the programcode further comprising code to: determine a placement in the userinterface of at least one of the plurality of content items based on awin share associated with the at least one of the plurality of contentitems.
 22. The non-transitory computer-readable storage medium of claim15, the program code further comprising code to: determine an orderingof two or more of the plurality of content items in the user interfacebased on a win share associated with each of the two or more contentitems.
 23. The non-transitory computer-readable storage medium of claim15, the program code further comprising code to: determine a frequencywith which at least one of the plurality of content items is madeavailable via the user interface based on a win share associated withthe at least one of the plurality of content items.
 24. Thenon-transitory computer-readable storage medium of claim 15, wherein theuser is a member of a sample of a user population, the program codefurther comprising code to: make one or more of the plurality of contentitems available to the user population based on win shares assignedusing the user sample.
 25. A non-transitory computer-readable storagemedium tangibly encoded with computer-executable program code, that whenexecuted by a computing device, performs a method comprising: making auser interface available over a network; making a plurality of contentitems available via the user interface, at least one content item of theplurality being made available based on a win share assigned to thecontent item, wherein a win share is determined using the program codewhich further comprises code to: track user interaction with the userinterface; recognizing a predetermined interaction of a user with theuser interface, said predetermined interaction identifies a time of thepredetermined interaction, the user and a content item with which theuser interacted; identifying a set of content items from the pluralityof content items with which the user interacted prior to thepredetermined interaction; and determining a share of a winning score toeach identified content item in said set of content items based on anumber of content items in said set, said determination furthercomprising assigning said share to each identified content item.
 26. Thenon-transitory computer-readable storage medium of claim 25, the programcode further comprising code to: determine a placement in the userinterface of at least one of the plurality of content items based on awin share associated with the at least one of the plurality of contentitems.
 27. The non-transitory computer-readable storage medium of claim25, the program code further comprising code to: determine an orderingof two or more of the plurality of content items in the user interfacebased on a win share associated with each of the two or more contentitems.
 28. The non-transitory computer-readable storage medium of claim25, the program code further comprising code to: determine a frequencywith which at least one of the plurality of content items is madeavailable via the user interface based on a win share associated withthe at least one of the plurality of content items.
 29. A computingdevice comprising: a processor; a non-transitory computer-readablestorage medium for tangibly storing thereon program logic for executionby the processor, the program logic comprising: tracking logic executedby the processor for tracking user interaction with a user interface;determining logic executed by the processor for recognizing apredetermined interaction of a user with the user interface, saidpredetermined interaction identifies a time of the predeterminedinteraction, the user and a content item with which the user interacted;identifying logic executed by the processor for identifying a set ofcontent items from a plurality of content items with which the userinteracted prior to the predetermined interaction; and determinationlogic executed by the processor for determining a share of a winningscore to each identified content item in said set of content items basedon a number of content items in said set, said determination furthercomprising assigning said share to each identified content item.
 30. Thecomputing device of claim 29, further comprising: assigning logicexecuted by the processor for assigning a share of a winning score toeach identified content item; and communication logic executed by theprocessor for distributing the winning score evenly across two or moreidentified content items.
 31. The computing device of claim 29, furthercomprising: assigning logic executed by the processor for assigning ashare of the winning score to each identified content item; andcommunication logic executed by the processor for distributing thewinning score unevenly across two or more identified content items. 32.The computing device of claim 31, further comprising: communicationlogic executed by the processor for distributing the winning scoreunevenly across two or more identified content items; determinationlogic executed by the processor for determining a weight for each of theidentified content items that is based on a timing of the user'sinteraction with the identified content item; and application logicexecuted by the processor for applying the identified content item'sweight to the winning score to determine a win share for the identifiedcontent item.
 33. The computing device of claim 32, further comprising:communication logic executed by the processor for distributing the winshare unevenly across two or more identified content items;determination logic executed by the processor for determining a weightfor each of the identified content items that is based on a newnesscharacteristic associated with the identified content item; andapplication logic executed by the processor for applying the identifiedcontent item's weight to a win share value to determine a win share forthe identified content item.
 34. The computing device of claim 29,further comprising: identifying logic executed by the processor foridentifying at least one of the plurality of content items to makeavailable via the user interface based on a win share associated withthe at least one of the plurality of content items.
 35. The computingdevice of claim 29, further comprising: determination logic executed bythe processor for determining a placement in the user interface of atleast one of the plurality of content items based on a win shareassociated with the at least one of the plurality of content items. 36.The computing device of claim 29, further comprising: determinationlogic executed by the processor for determining an ordering of two ormore of the plurality of content items in the user interface based on awin share associated with each of the two or more content items.
 37. Thecomputing device of claim 29, further comprising: determination logicexecuted by the processor for determining a frequency with which atleast one of the plurality of content items is made available via theuser interface based on a win share associated with the at least one ofthe plurality of content items.
 38. The computing device of claim 29,wherein the user is a member of a sample of a user population, saidprogram logic further comprising: communication logic executed by theprocessor for making one or more of the plurality of content itemsavailable to the user population based on win shares assigned using theuser sample.
 39. A computing device comprising: a processor; anon-transitory computer-readable storage medium for tangibly storingthereon program logic for execution by the processor, the program logiccomprising: user interface logic executed by the processor for making auser interface available over a network; communication logic executed bythe processor for making a plurality of content items available via theuser interface, at least one content item of the plurality being madeavailable based on a win share assigned to the content item; anddetermination logic executed by the processor for determining a winshare, said determination comprising: tracking user interaction with theuser interface; recognizing a predetermined interaction of a user withthe user interface, said predetermined interaction identifies a time ofthe predetermined interaction, the user and a content item with whichthe user interacted; and identifying a set of content items from theplurality of content items with which the user interacted prior to thepredetermined interaction; and determining a share of a winning score toeach identified content item in said set of content items based on anumber of content items in said set, said determination furthercomprising assigning said share to each identified content item.
 40. Thecomputing device of claim 39, further comprising: determination logicexecuted by the processor for determining a placement in the userinterface of at least one of the plurality of content items based on awin share associated with the at least one of the plurality of contentitems.
 41. The computing device of claim 39, further comprising:determination logic executed by the processor for determining anordering of two or more of the plurality of content items in the userinterface based on a win share associated with each of the two or morecontent items.
 42. The computing device of claim 39, further comprising:determination logic executed by the processor for determining afrequency with which at least one of the plurality of content items ismade available via the user interface based on a win share associatedwith the at least one of the plurality of content items.